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THE TIME-SPACE RELATIONSHIPS OF THE DATA POINTS (PIXELS) 
OF THE THEMATIC MAPPER AND THE MULTISPECTRAL SCANNER 

OR 

“THE MYTH OF SIMULTANEITY” 

Frederick Gordon, Jr. 

Goddard Space Flight Center 
Greenbelt, Maryland 


INTRODUCTION 

When speaking to users of satellite scanner data, it is always interesting to find how widespread is the 
notion which equates the temporal nature of that scanner data with the unit time aspect of photo- 
graphic data. I call this the “myth of simultaneity.” For most applications, equating the two may 
make little difference, but I believe it important that the user at least be aware of it. In this paper, 
the examples of the multispectral scanner (MSS) on Landsats 1 , 2, and 3 and the Thematic Mapper 
(TM) of Landsat D will be used to describe the concept and degree of non-simultaneity of scanning 
system data. For overall Landsat systems operation and performance refer to the references. This 
article will address itself primarily to the time-space aspects of scanner data acquisition and to those 
parts of the MSS and TM systems that are related to this phenomena. 

Let us briefly address the time aspect of “photographic” systems. In systems of this type, all data in 
the scene are acquired at the same point in time. It does not matter if the recording medium is film, 
the light sensitive surface of a vidicon tube, or any other photosensitive recording medium. The 
latter may be subsequently scanned for telemetry and/or processing purposes, but this has no essen- 
tial bearing on the time relationship among data points within the scene. Every part of the scene 
contains data that were frozen at a time zero, the time the scene was acquired. Landsats 1 and 2 
had three Return Beam Vidicon (RBV) cameras, each spectrally filtered for a different waveband, 
but all focused on the same area and shuttered simultaneously every 25 seconds. The vidicons were 
subsequently scanned sequentially, but the data acquired on each were for that same time of 
exposure. 

Landsat 3 uses a panchromatic two-camera system. The cameras are not exposed simultaneously, 
but the data acquired by either camera for a given scene are still all at a unique time. While in each 
case the data are scanned and telemetered at a period subsequent to exposure, every datum within 
a given scene still has the property of time-simultaneity. 

LANDSATS 1,2, and 3 

This brings us to the “myth of simultaneity” with regard to scanner data. Many users of Landsat 
MSS data have the intuitive feeling that the data they are working with have the same time quality as 
described above. In most cases this will make little difference in data-use, providing the mechanics 
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of scanning are known and the data processing system is adequate. However, the point of interest 
here is that in most scanner systems no datum that goes into making up a scene or image is acquired 
simultaneously with any other datum in that scene.* However, the time differences may be, and 
usually are, minute. In the case of the MSS of Landsats 1 , 2, and 3 these times may be as small as a 
fraction of a microsecond, but the time differences are finite and cumulative. The last datum in a 
Landsat scene will be taken close to 29 seconds after the first. Thus each of the 3 1 million or so 
pixels that go into making a 4 spectral band scene has been taken at a different time. Figure 1 is the 
layout of the optical fiber ends of the reflective bands and the ports of the thermal band relay optics 
in the focal plane of the Landsat MSS. The static equivalent ground dimensions are shown across and 
between the optical elements, as well as the equivalent number of spacecraft and in-band bytes. An 
in-band byte is equivalent in time to that necessary to measure one minor frame (MNFR); a minor 
frame is a single sampling of each of the 24+1 elements that comprise the optical matrix in the focal 
plane (1 MNFR = 9.958 microseconds). A spacecraft byte equates to the time necessary to take the 
measurement of any given optical element (0.39832 microseconds). These measurements are subse- 
quently amplified, digitized, etc. Descriptions of these operations can be found in the references, 
but are not needed to understand arguments presented here. Figure 2 is the time-distance layout of 
one 4-band line of the MSS optical matrix. The bottom line represents a series of pixel-equivalent 
areas along the line of scan of the MSS mirror.** Keep in mind that we are using an idealized situ- 
ation, where systematic perturbations such as spacecraft attitude and linearity of mirror sweep have 
been corrected, and where altitude is constant. Two directions of motion are involved relative to the 
focal plane matrix: the movement of the spacecraft roughly in the direction of the six lines per band 
and the motion of the scan of the mirror across the bands. For Figure 2 we take into consideration 
only the scan motion. At time Tq we read the output of sensor SI , the start of our minor frame 
measurement, in the sequence indicated in Figure 1 . 

If we allow 0.39832 microseconds between individual measurements (one spacecraft byte), S2 will be 
measured at T + 0.39832 microseconds. The cross scan velocity is calculated to be 5.612 meters per 
microsecond. The static equivalent ground distance (a non-moving mirror looking at a non-moving 
Earth) between centers of SI and S2 is (79+2) + (79+2) + 35 = 1 14 (based upon a nominal altitude of 
about 918 km).t However, in the 0.39832 microseconds between measurement S2 has moved about 
2 meters (0.39832 microseconds x 5.612 meters per microsecond). Thus, the actual center distance 
between the two viewed areas is 1 12 meters, which is equivalent to 2 pixels in the line of scan. So, 
when the measurement is made by detector S2, it is actually viewing a piece of ground that is 1 12 
meters west of the ground that had been measured 0.39832 microseconds before by SI . This would 
be pixel P-2 in Figure 2. 

In Figure 1, band 6 on that line is not sampled until 12 time-measurement units (spacecraft bytes) 
later. The distance that the scan has moved in this time interval is 12 x 0.39 x 5.6 ^ 26 meters. The 
static space distance between these 2 pixels is 79 + 35 + 79 + 57 = 250 meters. However, because 
the scan has moved 26 meters eastward during the time between the measurement on SI (band 4) 
and S13 (band 6), the sensor instantaneous field of view (IFOV) is looking at a piece of ground 


♦There is one aspect of the thematic mapper that is an exception to this generalization. This will be described later. 
**See Appendix A 

f Actual Landsat altitudes vary from 880 km 940 km, so that linear IFOV may vary from 76 m to 81 m. 
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224 meters (250-26) to the west of that viewed by SI within that same minor frame, or the equiva- 
lent of 4 pixels. On Figure 2 this would be pixel P-4. The detector that views band 7 on that line 
(S14) is measured one spacecraft byte later. Its space-time relationship to S13 is exactly the same as 
S2 was to SI, so it is 2 pixels to the west of S13, or 6 pixels (336 meters) to the west of P-0. Thus 
the measurement made by the four detectors in a single line during a single minor frame (in-band 
byte) were at four different locations (P-0, P-2, P-4, P-6). 

Within each band, for a given detector line, the pixels will be contiguous, moving eastward. However, 
if they were indexed by time, i.e., minor-frame by minor-frame (Tn), problems in spatial matching 
would arise. In the case of single-band black and white photoprints that were to be used to make 
color images you would have a 2 pixel difference between sequential bands. In order to correct this 
in digital tapes, pixels are added in front of each line of each band. In this way, the result is spatial 
alignment for the imagery, and the proper corrections made, computer compatible tapes (CCTs) are 
created (see Figure 3). The per-line algorithm for the number of pixels to be added in front of the 
band is N=8 - (2xn); where N-number of pixels to be inserted in front, n=number of the MSS band; 
n=l is the green band, etc. Actually, six pixels are added to each line in A- and X-format CCTs (so- 
called “raw” tapes), so that if 2 are added in front, the 4 would be added at the end. 

So far, only the single-line time relationships have been discussed, and it has been assumed that 
within a line the motion due to the rotation of the Earth is insignificant. We will continue to treat 
the latter point, as such, within a single sweep because during the 33 milliseconds required the 
185 km scan, the Earth has rotated only about 153 meters at the equator and (1 53 x cosine latitude) 

1 53 X .766 = 117 meters at 40 degrees latitude. This amounts to about 3 cm per pixel. 

In the line-to-line relationships, the simultaneity problem must again be addressed. Figure 1 shows 
that as we sample from line-to-line with a single band, there is a two measurement-time difference 
which is equal to 4.47 meters. There are 10 measurement times between line 1 and line 6 in any 
given band for a given minor frame, 22.4 meters or about 28 percent displacement. When the 
X-format CCTs and A-format CCTs (so-called raw data tapes) are made, this is ignored. However, 
when individual photo images are made (via the electron beam recorder or the laser beam recorder) 
this correction factor is included. Geometerically-corrected digital tapes (P-tapes) take this into con- 
sideration in the resampling process. 

It was stated above that relative movement of the target due to the Earth’s rotation could be ignored 
for a single sweep, but for the total scene (2,350 lines) this translational factor becomes significant. 

In the 1 85 km length of an MSS image, the time of travel along the orbital path is 28.6 seconds. In 
that time, the Earth has rotated a distance in kilometers equal to 0.4638 x cos latitude x 28.6, at a 
given line of latitude. At 40° north, this would be a 10.2 km displacement westward of the last line 
over the first. 

This is usually corrected in the image display systems by inserting pixels that effectively move lines 
westward. This is not a resampling operation. It only inserts an appropriate integral number of 
dummy pixels at the beginning and end of each line in conformance with some algorithm that 
ensures that the integrity of each 6-line sweep is maintained. The result of such an algorithm is 
shown in Table 1 . Input routines that make this correction usually remove the synthetic pixels, that 
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Table 1. 


LINE 

PIXELS 

LINE 

PIXELS 

UMBER 

INSERTED 

NUMBER 

INSERTED 

1 

2 

0 

495 

36 

0 

496 

36 

3 

0 

497 

36 

4 

0 

498 

36 

5 

0 

499 

36 

6 

0 

500 

36 

7 

0 

501 

36 

8 

0 

502 

36 

g 

0 

503 

36 

10 

0 

504 

36 

11 

0 

505 

36 

12 

0 

506 

36 

13 

0 

507 

36 

14 

0 

508 

36 

15 

0 

509 

36 

16 

0 

510 

36 

17 

0 

511 

37 

18 

0 



19 

1 

995 

72 

20 

1 

996 

72 

21 

1 

997 

73 

22 

1 

998 

73 

23 

1 

999 

73 

24 

1 

1000 

73 

25 

1 

1001 

73 

26 

1 

1002 

73 

27 

1 

1003 

73 

28 

1 

1004 

73 

29 

1 

1005 

73 

30 

1 

1006 

73 

31 

2 

1007 

73 

32 

2 

1008 

73 

33 

2 

1009 

73 

34 

2 

1010 

73 

35 

2 

1011 

73 

36 

2 

1012 

73 

37 

2 

1013 

73 

38 

2 

1014 

73 

39 

2 

1015 

74 

40 

2 

1016 

74 

41 

2 



42 

2 

2308 

168 

43 

3 

2309 

168 

44 

3 

2310 

168 



2311 

169 

480 

34 

2312 

169 

481 

35 

2313 

169 

482 

35 

2314 

169 

483 

35 

2315 

169 

484 

35 

2316 

169 

485 

35 

2317 

169 

486 

35 

2318 

169 

487 

35 

2B19 

169 

488 

35 

2320 

169 

489 

35 

2321 

169 

490 

35 

2322 

169 

491 

35 

2323 

170 

492 

35 

2324 

170 

493 

494 

36 

36 
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had been inserted in the CCTs during production to ensure equal line length, prior to insertion of 
skew corrective pixels. 

With the description of the fact of non-simultaneity we have assumed fairly ideal conditions and 
minimized perturbations such as nonlinearities in the mirror sweep and other factors that cause devi- 
ations from the desired space-time relationships. The main point to keep in mind relative to the MSS 
is that each pixel value is a separate, but time-space related measurement that must be reassembled 
to create a geographic scene. In the following section on the TM we will observe that these time- 
space relationships are even more protracted and their acquisition and reassemblement is even more 
complicated. 

LANDSAT-D THEMATIC MAPPER 

The thematic mapper (TM) performance represents changes from the Multispectral Scanner (MSS) 
that are different in both quality and quantity, and thus the time-space relationships are more com- 
plex. The number of bands (7) and the number of lines per band (16), plus auxiliary words, make a 
TM minor frame count of 102 (versus 25 for the MSS). But the most significant factors contributing 
to the added complexity in time-space relationship of pixels are that the sweep of the TM is active in 
both directions, and resulting in the timing and reading operations of the multiplexer which are quite 
different. These operational factors grew out of system performance requirements to provide suffi- 
cient dwell time to get an acceptable signal-to-noise ratio for each pixel measurement. The major 
optical performance difference is that the need to have the active sweep in both directions, precludes 
the use of the simple planar motion of the Scan Mirror across the path of flight to achieve the full 
raster effect that this achieves in the case of the MSS. You will recall in the MSS, using the scan 
mirror motion alone, plus active sweep in one direction only, allows each set of six lines in a sweep 
to be abutted to its immediate predecessor (see Figure 4). However, if we use this motion alone to 
collect data in both directions we get a situation such as shown in Figure 5a, in which part of the 
sweep covers the same area twice (overlap) and part will miss areas altogether (underlap). A solu- 
tion to this is shown in Figure 5b. Here the direction of sweep is perpendicular to the line of flight 
and each half-cycle. However, if this is to be achieved the sweep motion must have two things that 
cannot be achieved by a scan-mirror motion alone. First, the vector of the sweep ground trace must 
be displaced linearly opposite to the direction of flight during the sweep. Secondly, at the end of 
each half-sweep it must be displaced the equivalent distance of the cross-sweep width, in the direc- 
tion of flight. The vector diagram is as shown in Figure 5c. This is achieved by means of a Scan Line 
Corrector (SLC). (See Appendix B for a description of the optical path relationships of the Scan 
Mirror and the SLC). 

At this point let us introduce the TM focal plane layout of the optical detectors for the 7 bands. It 
is an “effective” focal plane, because there are actually two physical locations for the two parts of 
the detector system: first, the primary focal plane, which contains the uncooled silicon detectors 
for the first 4 reflective bands (1) 0.45-0.52 micrometer (blue) and (2) (0.52-0.60 micrometers and 
2.08-2.35 micrometers) require cooled (95 k) indium antimonide detectors. These are located phys- 
ically behind the primary focal plane, and along with the Mercury-Cadmium-Telluride (Hg CdTe) 
thermal IR detectors are on a separate refrigerated mount. These detectors are made to optically 
appear in the primary focal plane by means of relay optics. In Figures 6 and 7 note the relatively 
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VIEW OF DETECTORS AS PROJECTED ON PRIME FOCAL PLANE 
LOOKING FROM -Z TOWARDS +Z (OPTICAL SYSTEM COORDINATES) 


1FOV =42.5/irad 
= 0.00408 in. 



Figure 6. Detector Projection at Prime Focal Plane. 
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large ground distances between the bands and the staggered nature of the detectors within a band. 
Figure 7 shows the numbering arrangement within a band. Fiber optics were not used and the 
detectors themselves are mounted on the focal plane. Because of this and related reasons, the 
spacings are needed to help minimize cross talk. The mechanism and pattern of reading and trans- 
mitting the data is not commutated and converted in the same time-format as the MSS. Figure 8 
shows the minor frame sampling format for the TM multiplexer. The data is acquired by tracking 
and holding each one-half (odd or even) detector-set for each band at a specific time thus the data 
is not a continuous time stream as in the MSS. The TM has a time-set for each half minor frame 
(odd or even numbered detectors) that is “photographic” in concept; i.e. all the detectors in that 
time-set are tracked and held and subsequently read out for a specific time. 

The tracking period (dwell time) for each odd and even number detector set (all bands) actually is 
9.6 1 1 microseconds, but the hold/read time for each occurs at alternating 4.81 microsecond intervals. 
The velocity of the ground trace in the cross scan direction is 3.12 meters per microsecond. There- 
fore, there is a 15 meter (3.12m/usec x 4.81 usee) locational difference between the odd and the 
even detectors when they are measured. This one-half IFOV or pixel in each band set. As shown in 
Figure 7, the odd-even between pixel spacing is 2.5 IFOVs. This 0.5 pixel difference results from 
time of results in a ground locational difference of either 2 or 3 pixels within a given band for odd 
and even detector, depending on direction of the sweep. Referring to Figure 7 and visualizing the 
ground pattern of these detectors moving from left to right (west to east), it is evident that if the 
odd-numbered detectors are measured first, when the even-numbered detectors in a given band are 
measured, they will have moved a half pixel closer to the area over which the odd-numbered detect- 
ors had been read. Thus, the ground location difference would be 2 pixels (2.5— 0.5). If the sweep 
had been in the opposite direction (east to west), the even-numbered detectors would be spatially 
leading the odds, but the time sequence of measurement would be the same; i.e. odd numbers read 
first. The result is that for the east-west sweep, the even-numbered detectors would be effectively 
moving away from the area measured by the odd detectors. The result is that ground locational dis- 
tance in this case would be 3 pixels (2.5 + 0.5). Figure 9 is a pictorial representation of this. Not- 
ing from Figures 6 and 7 the various interpixel distances involved, it is evident that a given time (To) 
for the odd-numbered detectors, the specific pixel distances between band 1 and 2 is 750 meters. 

The same is true for band 2 to band 3, and band 3 to band 4. Band 4 and band 7 are 1 ,350 meters 
apart (45 IFOV), and band 7 and band 5 are 780 meters (26 pixels). Thus in a given minor frame, 
at time Tq, there is a fixed distance difference between the odd-numbered pixels of band 1 and those 
of band 5 (this encompasses the reflective or high resolution bands) of 4.38 kilometers. Before the 
odd-numbered detectors of band 5 look at the same ground pixel as band 1, they will go through 146 
minor frames and cover all the intervening pixel areas (or vice versa). The relationship of the even 
detectors to these odd detectors will, in a given minor frame, depend on the direction of sweep. In 
each successive sweep the time sequence-location relationship is just 180° out of phase. So, when 
one builds an image of the pixels and considers a serial time stream of data, the reverse sweeps must 
be effectively flipped so that the cross-track relationships become the same. From the point of 
view of non-simultaneity of measurement of single pixel areas and the construction of a total TM 
1 85 x 185 kilometer image (38 million pixels per reflective band), one can see the complexity of 
nonfixed time relationships for different locations within and among bands in a given TM scene. 
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The thermal infrared (THIR) pixel area is 16 times the size of the other bands (120 meters x 120 
meters). It is sampled at one-fourth the rate of the other bands (four minor frames to complete the 
sampling of all the thermal IR detectors). The interspatial relationships are as indicated in Figure 6. 

It was noted above for the TM that each 48-detector subset (odds or evens) in a given minor frame is 
frozen at a fixed time within that minor frame. The MSS individual detectors were commutated in 
the analog mode, and each detector within a minor frame was sampled at a different time. They 
were digitized sequentially after this serial commutation, and all measurements went through to 
some A/D converter. The TM’s larger number of detectors and the requirements for longer dwell 
times to improve the signal-to-noise ratio of the higher resolution, and narrower spectral bands do 
not allow this approach to be used in the Thematic Mapper. (The tracking time of the TM for each 
detector is 9.61 microseconds for one IFOV; as opposed to 0.398 microseconds for the individual 
MSS detectors.) During the subsequent hold period, the signal level of the detectors resulting from 
the tracking phase is read out by the multiplexer element supplied for each band. The eight detec- 
tor outputs from each band are read by a given half of a minor frame and are A/D converted by an 
individual submultiplexer. Subsequently, the output of all these submux multiplexers is serialized 
by the main digital multiplexer. The result for a given minor frame is a digital stream of data and 
associated telemetry for all of the odd numbered detectors which would have been acquired at time 
To, and for all of the even number detectors which would have been acquired at a time. To + 4.81 
seconds. Inasmuch as the planned aspect ratio of the TM cross-scan pixels of TM is 1:1; there is no 
double sampling of areas as with the MSS (as explained in Appendix A). Figure 10 shows the 
general layout of the TM multiplexer with its dual system. 

To summarize the factors involved in understanding the relationship among the Thematic Mapper 
pixels and those that have to be considered in constructing an image from the data points: 

1. The scanner sweep vectors that are 180° out of phase during each half-scan. All within- 
span relationships must take this into account. 

2. At a given time, the various reflective bands may be looking at areas as far as 4.38 kilo- 
meters apart. To view the same ground pixel area that has been viewed by band 1 , band 5 
must look at 146 intervening pixel areas over a period of 146 minor frames. Random jitter 
or attitude changes occurring during this period could preclude it from even looking at the 
same ground pixel area; or even being aware of it. 

3. The scan mirror-scan line corrector relationships must be known so that possible line-to-line 
overlap or underlap conditions can be accounted for and corrected. 

4. The only parameter of commonality for keeping track of the pixels is the time of 
acquisition-system operation relationships. These must be accurately maintained and under- 
stood for the 2 billion plus bits of data per scene in order for usable data to be evolved. 

SUMMARY 

The foregoing explanations are an attempt to explain the jigsaw puzzle nature of scanner data and 
how the puzzle must be assembled to make geographic sense of the data. The key element to the 
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assemblement of the pixels into data that is capable of being developed into information by the user, 
is time: time of acquisition and its relationship to the on-board processing and transmission system. 
The resultant scanner scene will be made up of millions of segments that have time as their locational 
key, but each pixel will have a time-space value that is different from every other pixel. 
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APPENDIX A 


For those who may not be familiar with the reason for the discrepancy between ground pixel equiv- 
alent size and that which might be expected from a square LIFOV, the following explanation is 
offered. Figure 2 shows the ground trace of measured segments as a line of segments or pixels 56 
meters x 79 meters. The actual instantaneous field of view of the individual MSS scanner detector 
element in the focal plane is the area on the ground that subtends an angular field of view of 86 
microradians (4.93 millidegrees). At a nominal altitude of 918 km the area IFOV is a square that is 
79 meters on a side. In Figure Ala, Tq represents this nominal field of view. The next time (T^ ) 
that a given detector element is sampled is one minor frame later (9.96 microseconds). In this inter- 
val, the relative motion of ground to the viewing detector in the direction of scan is 56 meters (scan 
velocity = 5.61 meters/microseconds) x 9.96 microseconds. Figure Alb shows that this results in 
an area between pixels of 23 x 79 meters (1817 m^) that is sampled twice. Figure Ale represents a 
segment of the cross-scan ground line of Figure 2. The measurement pixel which is 56 x 79 meters 
(4424 m^) has a portion of it on each side (1 1.5 m x 79 m = 908.5) that is sampled twice. This is 
equal to overlap area indicated in Alb. This overlap area is about 41 percent of the total measure- 
ment pixel area (1817-^ 4424). This is usually described as having 1 : 1 .4 1 aspect ratio in the direc- 
tion of scan. In the perpendicular direction (roughly the direction of orbital motion) the field of 
views are designed to abut each other, that is they nominally have no overlap. In this direction, the 
aspect ratio is 1 :1. In sampling theory there are desirable factors to oversampling. However, other 
system design considerations may preclude its use, as was the case in the oribtal direction of Land- 
sat. The thematic mapper of Landsat-D is designed to have a 1 : 1 aspect ratio in both directions. 
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APPENDIX B 


The general function of the Scan Mirror Assembly (SMA) in the TM is identical with that of the MSS, 
to provide a continually changing view by the sensors across the direction of flight of the spacecraft. 
This action, coupled with the orbital motion the spacecraft itself, provides the matrix that goes into 
providing a 2-dimensional scene. In the MSS, the SMA motion coupled with flight motion was ade- 
quate to accomplish this because you have active sweep in one direction only. As explained in the 
text, this is not so with the TM. The scan motion must also be modified to achieve an effective cross- 
track motion that is perpendicular to the line of flight. The means of accomplishing this is the scan 
line corrector optics (SLC). Figure B1 is a simplified diagram of the optics of the TM. Figure B2 is 
a schematic representation of the action of the SMA. The diagram is facing northward, and the direc- 
tion of flight would be outward from the page. If wej consider the ground elements Aj^ as series of 
pixels observed by a given detector D1 , we see that as the SMA moves through an angle a, the ground 
trace moves through an angle of 2a = B. Unless altered, this motion would describe the ground pat- 
tern shown in Figure 4 of the main text, except that it would have 16 lines for the TM. If we insert 
a set of parallel minors whose axis of rotation is roughly perpendicular to that of the SMA, we intro- 
duce a vector motion that translates the observed pixel northward as we go in either direction across 
the scan. The effect is that a given detector is following an across-scan trace that is the result of these 
two motions. Combining Figures B2 and B3 and looking backward from the detector, it is evident 
that the action of the SLC effectively is moving the point of vision in the plane of the SMA mirror 
into the paper in Figure B1 as A„ sweeps from A1 to A2 or vice versa. (For a more detailed explana- 
tion of the optics see ref. 6.) Note that as the active motion of the SLC mirrors must always be in 
the same direction no matter what the sweep direction of the SMA mirror, the effective angular ro- 
tation rate of the SLC must be twice that of the SMA. 
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Figure B-1. Optical System Elements. 


SCAN MIRROR 



Figure B-2 . 
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Figure B-3 Scan Line Corrector. 
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